Rotation Recovery from Spherical Images without Correspondences

This paper addresses the problem of rotation estimation directly from images defined on the sphere and without correspondence. The method is particularly useful for the alignment of large rotations and has potential impact on 3D shape alignment. The foundation of the method lies in the fact that the spherical harmonic coefficients undergo a unitary mapping when the original image is rotated. The correlation between two images is a function of rotations and we show that it has an SO(3)-Fourier transform equal to the pointwise product of spherical harmonic coefficients of the original images. The resolution of the rotation space depends on the bandwidth we choose for the harmonic expansion and the rotation estimate is found through a direct search in this 3D discretized space. A refinement of the rotation estimate can be obtained from the conservation of harmonic coefficients in the rotational shift theorem. A novel decoupling of the shift theorem with respect to the Euler angles is presented and exploited in an iterative scheme to refine the initial rotation estimates. Experiments show the suitability of the method for large rotations and the dependence of the method on bandwidth and the choice of the spherical harmonic coefficients

Deformable Shape Completion with Graph Convolutional Autoencoders

The availability of affordable and portable depth sensors has made scanning objects and people simpler than ever. However, dealing with occlusions and missing parts is still a significant challenge. The problem of reconstructing a (possibly non-rigidly moving) 3D object from a single or multiple partial scans has received increasing attention in recent years. In this work, we propose a novel learning-based method for the completion of partial shapes. Unlike the majority of existing approaches, our method focuses on objects that can undergo non-rigid deformations. The core of our method is a variational autoencoder with graph convolutional operations that learns a latent space for complete realistic shapes. At inference, we optimize to find the representation in this latent space that best fits the generated shape to the known partial input. The completed shape exhibits a realistic appearance on the unknown part. We show promising results towards the completion of synthetic and real scans of human body and face meshes exhibiting different styles of articulation and partiality.Comment: CVPR 201

Motion Estimation Using a Spherical Camera

Robotic navigation algorithms increasingly make use of the panoramic field of view provided by omnidirectional images to assist with localization tasks. Since the images taken by a particular class of omnidirectional sensors can be mapped to the sphere, the problem of attitude estimation arising from 3D motions of the camera can be treated as a problem of estimating the camera motion between spherical images. This problem has traditionally been solved by tracking points or features between images. However, there are many natural scenes where the features cannot be tracked with confidence. We present an algorithm that uses image features to estimate ego-motion without explicitly searching for correspondences. We formulate the problem as a correlation of functions defined on the product of spheres S2 × S2 which are acted upon by elements of the direct product group SO(3) × SO(3). We efficiently compute this correlation and obtain our solution using the spectral information of functions in S2 × S2

Planar Ego-Motion Without Correspondences

General structure-from-motion methods are not adept at dealing with constrained camera motions, even though such motions greatly simplify vision tasks like mobile robot localization. Typical ego-motion techniques designed for such a purpose require locating feature correspondences between images. However, there are many cases where features cannot be matched robustly. For example, images from panoramic sensors are limited by nonuniform angular sampling, which can complicate the feature matching process under wide baseline motions. In this paper we compute the planar ego-motion of a spherical sensor without correspondences. We propose a generalized Hough transform on the space of planar motions. Our transform directly processes the information contained within all the possible feature pair combinations between two images, thereby circumventing the need to isolate the best corresponding matches. We generate the Hough space in an efficient manner by studying the spectral information contained in images of the feature pairs, and by re-treating our Hough transform as a correlation of such feature pair images

Correspondenceless Structure from Motion

We present a novel approach for the estimation of 3D-motion directly from two images using the Radon transform. The feasibility of any camera motion is computed by integrating over all feature pairs that satisfy the epipolar constraint. This integration is equivalent to taking the inner product of a similarity function on feature pairs with a Dirac function embedding the epipolar constraint. The maxima in this five dimensional motion space will correspond to compatible rigid motions. The main novelty is in the realization that the Radon transform is a filtering operator: If we assume that the similarity and Dirac functions are defined on spheres and the epipolar constraint is a group action of rotations on spheres, then the Radon transform is a correlation integral. We propose a new algorithm to compute this integral from the spherical Fourier transform of the similarity and Dirac functions. Generating the similarity function now becomes a preprocessing step which reduces the complexity of the Radon computation by a factor equal to the number of feature pairs processed. The strength of the algorithm is in avoiding a commitment to correspondences, thus being robust to erroneous feature detection, outliers, and multiple motions

Fully Automatic Registration of 3D Point Clouds

We propose a novel technique for the registration of 3D point clouds which makes very few assumptions: we avoid any manual rough alignment or the use of landmarks, displacement can be arbitrarily large, and the two point sets can have very little overlap. Crude alignment is achieved by estimation of the 3D-rotation from two Extended Gaussian Images even when the data sets inducing them have partial overlap. The technique is based on the correlation of the two EGIs in the Fourier domain and makes use of the spherical and rotational harmonic transforms. For pairs with low overlap which fail a critical verification step, the rotational alignment can be obtained by the alignment of constellation images generated from the EGIs. Rotationally aligned sets are matched by correlation using the Fourier transform of volumetric functions. A fine alignment is acquired in the final step by running Iterative Closest Points with just few iterations

Radon-based Structure from Motion Without Correspondences

We present a novel approach for the estimation of 3Dmotion directly from two images using the Radon transform. We assume a similarity function defined on the crossproduct of two images which assigns a weight to all feature pairs. This similarity function is integrated over all feature pairs that satisfy the epipolar constraint. This integration is equivalent to filtering the similarity function with a Dirac function embedding the epipolar constraint. The result of this convolution is a function of the five unknownmotion parameters with maxima at the positions of compatible rigid motions. The breakthrough is in the realization that the Radon transform is a filtering operator: If we assume that images are defined on spheres and the epipolar constraint is a group action of two rotations on two spheres, then the Radon transform is a convolution/correlation integral. We propose a new algorithm to compute this integral from the spherical harmonics of the similarity and Dirac functions. The resulting resolution in the motion space depends on the bandwidth we keep from the spherical transform. The strength of the algorithm is in avoiding a commitment to correspondences, thus being robust to erroneous feature detection, outliers, and multiple motions. The algorithm has been tested in sequences of real omnidirectional images and it outperforms correspondence-based structure from motion